Iteration Complexity of Fixed-Step Methods by Nesterov and Polyak for Convex Quadratic Functions

Abstract This note considers the momentum method by Polyak and accelerated gradient Nesterov, both without line search but with fixed step length applied to strongly convex quadratic functions assuming that exact gradients are used appropriate upper lower bounds for extreme eigenvalues of Hessian matrix known. Simple 2-d-examples show Euclidean distance iterates optimal solution is non-monotone. In this context, an explicit bound derived on number iterations needed guarantee a reduction factor $$\epsilon $$ ϵ . For methods, up constant factor, it complements earlier asymptotically results method, establishes another link Nesterov’s method.